Cache memories are used in CPU architectures to improve the access time of CPU requested instructions and operand data. Typically, a cache memory is a relatively low capacity, high-speed memory intimately coupled to a single CPU. Such cache memories are described as being `local` with respect to their immediately associated CPU. Cache memories are thus quite distinct from the substantially larger, though significantly slower, main store memory unit that is typically implemented as a device common to all CPUs of the data processing system.
A number of considerations are involved in the utilization of cache memories Conventional cache memories provide selective, temporary storage of lines of instructions and operand data following transfer from the mainstore to the CPU. Closely subsequent requests for cache stored instructions or operand data are satisfied locally by accesses of the current copy present in the local cache memory. The determination of whether any particular CPU request can be satisfied by accessing the local cache memory is made by a CPU Storage Unit (SU) commonly including a cache controller, intimately also coupled to the local cache. Typically, the cache controller maintains a tag list identifying the instructions and data lines present in its cache. Where a request cannot be satisfied by a cache memory access, the storage unit passes the request to the mainstore memory unit for subsequent completion. A copy of the data line containing the requested instructions or operand data is placed in the local cache memory of the CPU. Thus, the cache memory can be described as having a `CPU` side interface, where instructions and operand data are passed back and forth between the CPU and cache memory, and a `mainstore side` interface that is coupled between the cache memory and the main store memory unit for the transfer of instructions and operand data lines.
Given the finite capacity of the cache memory, typical CPU requests that require instructions and operand data to be sourced from the main store memory unit further require the prior transfer of a data line from the cache memory to main store to free up adequate storage space for the currently requested instructions or data. This management of the local cache memory contents is typically considered a low level, or fundamental, system integrity function.
A consequence of this system integrity function is that satisfaction of the CPU instruction or operand memory line request is significantly delayed by the swapping out of a cache line to mainstore. The swap delay can be reduced by temporarily buffering the swapped out memory line rather than writing it directly to mainstore. Once buffered, the requested memory line can be transferred into the cache. However, the temporarily buffered data line must then be immediately written out to mainstore to ensure its availability potentially for the immediately subsequent CPU request. Thus, the swap out delay is again encountered whenever there are closely successive CPU requests for new memory lines. Consequently, the CPU benefits from a significantly higher CPU-side cache instruction and operand data bus bandwidth whenever requests can be directly satisfied by accesses to the local cache memory. However, the CPU is generally limited to the significantly lower mainstore memory-side cache data line bus bandwidth limit where instruction or operand data is only available from mainstore.
Another complication in the use of cache memories is that typical mainframe data processing architectures utilize a segmented or pipelined processing structure for the simultaneous, though staggered, execution of instructions and the processing of respective operand data. As a result, access conflicts arise between the various pipeline segments when trying to retrieve instruction and operand data present in the cache memory. A solution to this problem is obtained by providing separate caches for instructions and operand data. Such split-cache architectures allow the instruction execution pipeline segments to compete with their operand data manipulation counterparts for access to the respective instruction and operand data caches.
Unfortunately, a significant problem arises by the use of a split cache architecture. Typically, all requests for instruction and operand data from the main store memory unit are satisfied by the transfer of an entire memory line to the requesting CPU. Each line will contain the requested instruction or operand data embedded in the line along with all of the instructions and operand data sharing the same line address. Where a request is made for operand data, the corresponding memory line is stored specifically to the operand data cache. Conversely, an instruction request results in the storage of the memory line in the instruction cache. Consequently, a copy of specific operand data may appear in both the instruction and operand data caches. This gives rise to a data integrity problem. If operand data is modified by the CPU and stored back to the operand data cache, incorrect operation of the CPU will result if the instruction cache copy of the operand data is subsequently accessed as an integral part of an executable instruction. Therefore, the data integrity maintenance function of split cache data processing systems must either invalidate or write out all memory lines sharing the request specified line address to the mainstore memory unit before satisfying a request for modifiable operand data. This insures that the mainstore memory is properly updated and that only one modifiable copy of the memory line requested exists outside of mainstore. Similarly, whenever an instruction memory line request must be satisfied, where that memory line already exists as a modifiable memory line in an operand data cache memory, the operand data memory line must be either written out to the mainstore memory unit, if modified, or invalidated to preclude modification and ensure that the single most recently modified memory line is returned to the requesting CPU.
Normally, where the modifiable application program data is maintained distinctly separate from the executable code portion of the application program, the split cache data integrity function is rarely invoked. Thus, there is little impact on system performance. However, a condition typically referred to as `thrashing` will occur where the application program instruction code and potentially modifiable operand data are closely intermixed, such as to both occur frequently in memory lines of the application program. That is, a memory line containing both instructions and operand data may be fetched and copied to the instruction cache only to be immediately invalidated to allow a subsequent modifiable operand data request to be satisfied from the same memory line. A closely subsequent instruction request to execute the same or next instruction will cause the modified memory line to be written back out to main store only to then be fetched back to the instruction cache memory. These steps may occur repeatedly and in close order particularly where the application program calls for the execution of a short program execution loop. Consequently, a significant and distinctly undesirable CPU performance degradation will occur.